路端多源数据空间一致性数据集构建及评估方法研究

doi:10.16451/j.cnki.issn1003-6059.202311007

摘要
图/表
参考文献(0)
相关文章 (1)

全文: PDF (6855 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要

构建多源传感数据空间一致性是路端多模态数据融合的基础,在车路协同及路端智能中发挥重要作用.然而,现有的路端多模态数据集主要侧重于目标检测等识别类任务的研究,缺少多源传感器之间的多种空间变换信息,不足以支撑路端多源数据空间一致性问题的研究.因此,文中构建一个专门用于路端多源数据空间一致性问题研究的数据集——InfraCalib(https://github.com/chenzhiwei888/InfraCalib-Dataset).数据集共包含23万多帧图像与点云数据,由两个路端智能移动设备采集,覆盖场景、模态、光照、设备空间位置及传感器姿态等多样变化.通过匹配特征关键点对关联多模态数据,构建PnP(Perspective-n-Point)问题,并利用最小重投影误差法解算外参矩阵,作为近似真值标签.最后,在InfraCalib数据集上进行经典特征匹配算法的实验分析,并讨论多源传感器外参标定的量化评估指标.

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	陈志伟
	张皓霖
	严宇宸
	陈仕韬

关键词 ：多源数据空间一致性, 车路协同, 空间变换, 最小重投影误差, 评估指标

Abstract：

The spatial consistency of multi-source sensing data is established as the foundation for the fusion of roadside multi-modal data, playing a crucial role in vehicle-to-infrastructure and roadside intelligence. However, existing roadside multi-modal datasets predominantly focus on recognition tasks such as object detection, lacking various spatial transformation information between multi-source sensors. This deficiency hinders the research into the spatial consistency problem of multi-source data at the roadside. Therefore, a dataset specifically designed for the study of the spatial consistency problem in roadside multi-source data is constructed in this paper-InfraCalib(https://github.com/chenzhiwei888/InfraCalib-Dataset). The dataset comprises over 230,000 frames of images and point cloud data, collected by two roadside smart mobile devices, covering diverse changes in scenes, modalities, lighting, device spatial positions and sensor postures. By matching feature key point pairs to correlate multi-modal data, a perspective-n-point(PnP) problem is constructed, and the extrinsic parameter matrix is solved using the minimum reprojection error method, serving as an approximate ground truth label. Finally, an experimental analysis of the classic feature matching algorithm is conducted on the InfraCalib dataset, and the discussion revolves around the quantitative evaluation indicators for the calibration of external parameters in multi-source sensors.

Key words： Multi-source Data Spatial Consistency Vehicle-to-Infrastructure Spatial Transformation Minimum Reprojection Error Evaluation Indicator

收稿日期: 2023-10-10

ZTFLH:

TP181

基金资助:

国家重点研发计划项目(No.2022YFB2502900)、国家自然科学基金项目(No.62088102)资助

通讯作者: 陈仕韬,博士,助教,主要研究方向为无人驾驶.E-mail:chenshitao@xjtu.edu.cn.

作者简介: 陈志伟,硕士研究生,主要研究方向为智能交通.E-mail:3121155023@stu.xjtu.edu.cn.张皓霖,博士研究生,主要研究方向为无人驾驶、车路协同.E-mail:zhanghaolin@xjtu.edu.cn.严宇宸,博士研究生,主要研究方向为多传感器的标定与感知.E-mail:yanyuchen@stu.xjtu.edu.cn.

引用本文:

陈志伟, 张皓霖, 严宇宸, 陈仕韬. 路端多源数据空间一致性数据集构建及评估方法研究[J]. 模式识别与人工智能, 2023, 36(11): 1041-1058. CHEN Zhiwei, ZHANG Haolin, YAN Yuchen, CHEN Shitao. Construction of Roadside Multi-source Data Space Consistency Dataset and Research on Evaluation Methods. Pattern Recognition and Artificial Intelligence, 2023, 36(11): 1041-1058.

链接本文:

http://manu46.magtech.com.cn/Jweb_prai/CN/10.16451/j.cnki.issn1003-6059.202311007 或 http://manu46.magtech.com.cn/Jweb_prai/CN/Y2023/V36/I11/1041

[1] CHEN C Y, SEFF A, KORNHAUSER A, et al. DeepDriving: Lear-ning Affordance for Direct Perception in Autonomous Driving // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2015: 2722-2730.
[2] XU H Z, GAO Y, YU F, et al. End-to-End Learning of Driving Models from Large-Scale Video Datasets // Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 3530-3538.
[3] BOJARSKI M, TESTA D D, DWORAKOWSKI D, et al. End to End Learning for Self-Driving Cars[C/OL]. [2023-09-16]. https://arxiv.org/pdf/1604.07316v1.pdf
[4] ONDRUŠKA P, POSNER I. Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks. Proceedings of the AAAI Confe-rence on Artificial Intelligence, 2016, 30(1): 3361-3367.
[5] CUI Y D, CHEN R, CHU W B, et al. Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review. IEEE Transactions on Intelligent Transportation Systems, 2021, 23(2): 722-739.
[6] LU N, CHENG N, ZHANG N, et al. Connected Vehicles: Solu-tions and Challenges. IEEE Internet of Things Journal, 2014, 1(4): 289-299.
[7] LUO R C, YIH C C, SU K L.Multisensor Fusion and Integration: Approaches, Applications, and Future Research Directions. IEEE Sensors journal, 2002, 2(2): 107-119.
[8] LOWE D G.Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision, 2004, 60: 91-110.
[9] RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: An Efficient Alternative to SIFT or SURF // Proc of the International Conference on Computer Vision. Washington, USA: IEEE, 2011: 2564-2571.
[10] BAY H, TUYTELAARS T, VAN GOOL L.SURF: Speeded up Robust Features // Proc of the 9th European Conference on Computer Vision. Berlin, Germany: Springer, 2006: 404-417.
[11] FENG M D, HU S X, ANG M H, et al. 2D3D-Matchnet: Lear-ning to Match Keypoints Across 2D Image and 3D Point Cloud // Proc of the International Conference on Robotics and Automation. Washington, USA: IEEE, 2019: 4790-4796.
[12] LOWE D G.Object Recognition from Local Scale-Invariant Features // Proc of the 7th IEEE International Conference on Compu-ter Vision. Washington, USA: IEEE, 1999. DOI: 10.1109/ICCV.1999.790410.
[13] ZHONG Y.Intrinsic Shape Signatures: A Shape Descriptor for 3D Object Recognition // Proc of the IEEE 12th International Confe-rence on Computer Vision Workshops. Washington, USA: IEEE, 2009: 689-696.
[14] LI J X, LEE G H.DeepI2P: Image-to-Point Cloud Registration via Deep Classification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 15955-15964.
[15] BESL P J, MCKAY N D.A Method for Registration of 3-D Shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14(2): 239-256.
[16] BIBER P, STRAβER W. The Normal Distributions Transform: A New Approach to Laser Scan Matching // Proc of the IEEE/RSJ International Conference on Intelligent Robots and Systems. Washing-ton, USA: IEEE, 2003: 2743-2748.
[17] YANG J L, LI H D, JIA Y D.Go-ICP: Solving 3D Registration Efficiently and Globally Optimally // Proc of the IEEE Internatio-nal Conference on Computer Vision. Washington, USA: IEEE, 2013: 1457-1464.
[18] LU W X, WAN G W, ZHOU Y, et al. DeepVCP: An End-to-End Deep Neural Network for Point Cloud Registration // Proc of the IEEE/CVF International Conference on Computer Vision. Wa-shington, USA: IEEE, 2019: 12-21.
[19] YEW Z J, LEE G H.RPM-Net: Robust Point Matching Using Learned Features // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 11821-11830.
[20] LI J X, LEE G H.USIP: Unsupervised Stable Interest Point Detection from 3D Point Clouds // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 361-370.
[21] YEW Z J, LEE G H.3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 630-646.
[22] ZENG A, SONG S,NIEβNER M, et al. 3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 199-208.
[23] GOJCIC Z, ZHOU C F, WEGNER J D, et al. The Perfect Match: 3D Point Cloud Matching with Smoothed Densities // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 5540-5549.
[24] TANG D Q, HU T J, SHEN L C, et al. AprilTag Array-Aided Extrinsic Calibration of Camera-Laser Multi-sensor System. Robotics and Biomimetics, 2016. DOI: 10.1186/s40638-016-0044-0.
[25] DHALL A, CHELANI K, RADHAKRISHNAN V, et al. LiDAR-Camera Calibration Using 3D-3D Point Correspondences[C/OL].[2023-09-16]. https://arxiv.org/abs/1705.09785.
[26] LI X C, XIAO Y X, WANG B B, et al. Automatic Targetless LiDAR-Camera Calibration: A Survey. Artificial Intelligence Review, 2023, 56(9): 9949-9987.
[27] ZHANG Z.A Flexible New Technique for Camera Calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334.
[28] RICHARDSON A, STROM J, OLSON E.AprilCal: Assisted and Repeatable Camera Calibration // Proc of the IEEE/RSJ International Conference on Intelligent Robots and Systems. Washington, USA: IEEE, 2013: 1814-1821.
[29] YE X Q, SHU M, LI H Y, et al. Rope3D: The Roadside Perception Dataset for Autonomous Driving and Monocular 3D Object Detection Task // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 21309-21318.
[30] YU H B, LUO Y Z, SHU M, et al. DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection // Proc of the IEEE/CVF Conference on Computer Vision and Pa-ttern Recognition. Washington, USA: IEEE, 2022: 21329-21338.
[31] YU H B, YANG W X, RUAN H Z, et al. V2X-Seq: A Large-Scale Sequential Dataset for Vehicle-Infrastructure Cooperative Perception and Forecasting // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2023: 5486-5495.
[32] CREβ C, ZIMMER W, STRAND L, et al. A9-Dataset: Multisen-sor Infrastructure-Based Dataset for Mobility Research // Proc of the IEEE Intelligent Vehicles Symposium. Washington, USA: IEEE, 2022: 965-970.
[33] GEIGER A, LENZ P, URTASUN R.Are We Ready for Autonomous Driving? the KITTI Vision Benchmark Suite // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2012: 3354-3361.
[34] CAESAR H, BANKITI V, LANG A H, et al. nuScenes: A Multimodal Dataset for Autonomous Driving // Proc of the IEEE/CVF conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 11618-11628.
[35] MEI J R, ZHU A Z, YAN X C, et al. Waymo Open Dataset: Pa-noramic Video Panoptic Segmentation // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2022: 53-72.
[36] CHANG M F, LAMBERT J, SANGKLOY P, et al. Argoverse: 3D Tracking and Forecasting with Rich Maps // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 8740-8749.
[37] MAO J G, NIU M Z, JIANG C H, et al. One Million Scenes for Autonomous Driving: ONCE Dataset[C/OL].[2023-09-16]. https://arxiv.org/pdf/2106.11037.pdf.
[38] P?IBYL B, ZEM?ÍK P, ?ADÍK M. Absolute Pose Estimation from Line Correspondences Using Direct Linear Transformation. Computer Vision and Image Understanding, 2017, 161: 130-144.
[39] RANGANATHAN A. The Levenberg-Marquardt Algorithm[C/OL]. [2023-09-16]. http://www.ananth.in/docs/lmtut.pdf.
[40] GAO X S, HOU X R, TANG J L, et al. Complete Solution Classification for the Perspective-Three-Point Problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(8): 930-943.
[41] LEPETIT V, MORENO-NOGUER F, FUA P.EPnP: An Accurate O(n) Solution to the PnP Problem. International Journal of Computer Vision, 2009, 81: 155-166.
[42] PENATE-SANCHEZ A, ANDRADE-CETTO J, MORENO-NOGUER F.Exhaustive Linearization for Robust Camera Pose and Focal Length Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(10): 2387-2400.
[43] FISCHLER M A, BOLLES R C.Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Communications of the ACM, 1981, 24(6): 381-395.
[44] VISWANATHAN D G.Features from Accelerated Segment Test(FA-ST) [C/OL]. [2023-09-16].https://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/AV1011/AV1FeaturefromAcceleratedSegmentTest.pdf.
[45] CALONDER M, LEPETIT V, STRECHA C, et al. BRIEF: Binary Robust Independent Elementary Features // Proc of the 11th European Conference on Computer Vision. Berlin, Germany: Sprin-ger, 2010: 778-792.